1. Field of the Invention
The present invention relates to an image processing method and an image processing device for performing an image transformation operation, and to an imaging apparatus using the image processing device.
2. Description of the Related Art
When an image transformation operation such as correction of a distorted image, rotation of an image, and change of an image to an overhead view is performed in conventional camera systems such as in-vehicle cameras, a method using a lookup table in which relation between coordinate values of pixels of an input image and coordinate values of pixels of an output image is preliminarily stored is used to perform the image transformation operation at a high speed. However, since recent image sensors have a great number of pixels, a memory having a large memory capacity has to be used for storing such a lookup table. In addition, it is necessary to prepare a lookup table for each image transformation operation. Since a variety of image transformation operations are needed recently, the number of lookup tables used increases, thereby increasing the costs of the camera systems.
In attempting to solve the problem, there is a proposal for an image processing method of transforming an input image to an output image using a lookup table, wherein the output image is segmented into multiple blocks, and coordinate values of pixels of the input image corresponding to vertex pixels of each of the multiple blocks constituting the output image are stored in the lookup table. When a coordinate value of a pixel of the input image corresponding to a focused pixel of the output image are not present on the lookup table, the coordinate values are calculated using a bilinear interpolation processing to decrease the size of memory for storing the lookup table, and to perform a variety of image transforming operations at a high speed using a device having a simple configuration.
In the method, for example, when it is desired that a portion of a captured image (i.e., input image) is displayed while enlarged (i.e., when digital zooming is performed), the output image is segmented into multiple blocks (e.g., 6×4 blocks in an image illustrated in FIG. 13), each of which has pixels of (a) pieces in the X-direction and pixels of (b) pieces in the Y-direction as illustrated in FIG. 13. In addition, the method uses a lookup table, in which coordinate values of pixels in the input image corresponding to vertex pixels of each block of the output image (e.g., 7×5 vertex pixels in the image illustrated in FIG. 13) are stored. In FIG. 13, reference numerals 1-35 denote the numbers of the vertex pixels (hereinafter referred to as a vertex pixel number) of the output image. The coordinate value of a pixel in the input image corresponding to one of the vertex pixels of the output image is stored in the lookup table.
When a focused pixel of the output image is identical to one of the vertex pixels, the coordinate value of the corresponding pixel of the input image stored at an address in the lookup table, which has the number of the one of the vertex pixels, is used. By contrast, when the focused pixel of the output image is not identical to each of the vertex pixels, the coordinate values of four vertex pixels of the input image corresponding to four vertex pixels of one of the blocks including the focused pixel are obtained from the lookup table, and the coordinate values of a pixel in the input image corresponding to the focused pixel in the output image are calculated by performing a linear interpolation processing using the coordinate values of the four vertex pixels. In this regard, the corresponding pixel of the input image is output as the focused pixel of the output image. When there is no pixel in the input image corresponding to the focused pixel of the output image, the pixel value of the focused pixel in the output image is determined by performing a linear interpolation processing on four pixels surrounding a point having the determined coordinate values.
By using the image processing method mentioned above, complex image transformation processings (such as digital zooming images illustrated in FIG. 13) can be realized using a small-sized lookup table. However, it is not easy for the image processing method to output a picture by picture (P by P) image, for example, by displaying both side portions of a captured image side by side (such as an image illustrated in FIG. 14B), because there is a discontinuous portion in the center of the output image. In addition, it is not easy for the image processing method to output a picture in picture (P in P) image, in which a second image is displayed in a first image, because there is a discontinuous portion at the boundary between the first and second images.
FIG. 14A is a schematic view illustrating a captured image, and FIG. 14B is a schematic view illustrating a picture by picture image prepared by using the captured image. Specifically, a portion A on the left side of the captured image illustrated in FIG. 14A and a portion B on the right side of the captured image are displayed side by side while enlarged (i.e., digital zooming) as illustrated in FIG. 14B. In this regard, the number of blocks (6×4 blocks) of the output image is the same as that in the output image illustrated in FIG. 13B. It can be understood from comparison of FIG. 14 with FIG. 13 that since the number of vertex data to be referred to at the discontinuous portion (i.e., the central portion) in the image illustrated in FIG. 14B is greater than that in the image illustrated in FIG. 13B, the number of vertex points whose information is stored in a lookup table is greater than that in the image illustrated in FIG. 13B. Specifically, in the image illustrated in FIG. 13B, the number of vertex points whose information is stored in a lookup table is 35 (i.e., 7×5) whereas the number of vertex points whose information is stored in a lookup table is 40 (i.e., 4×5+4×5) in the image illustrated in FIG. 14B. In a method of calculating coordinate values of pixels in an input image corresponding to pixels in an output image by performing a linear interpolation processing using a lookup table, it is necessary to perform complex processings at a discontinuous portion. Namely, it is not easy for the method to perform such complex processings.
For these reasons, the present inventor recognized that there is a need for an image processing method, by which coordinate values of pixels in an input image corresponding to pixels in an output image can be easily obtained by performing a linear interpolation processing or the like using a lookup table even when the output image has a discontinuous portion.